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Abstract 



■ It has been shown by Alon et al. that the so-called 'all-pairs shortest-path' problem can be 
L/~) solved in 0((MV) 2,688 log 3 (V)) for graphs with V vertices, with integer distances bounded by 
C~ — M. We solve the more general problem for graphs in R (assuming no negative cycles), with 
G\ expected-case running time 0(V 2 5 \og{V)). While our result appears to violate the Q(V ) 
5 'requirement of "Funny Matrix Multiplication" (due to Kerr), we find that it has a sub-cubic 

.expected time solution subject to reasonable conditions on the data distribution. The expected 
i-H time solution arises when certain sub-problems are uncorrelated, though we can do better/worse 
0*N than the expected-case under positive/negative correlation (respectively). Whether we observe 

-positive/negative correlation depends on the statistics of the graph in question. In practice, 

bur algorithm is significantly faster than Floyd-Warshall, even for dense graphs. 
• i-H , 

X 

1*-h Problem Definition 

03 ' 



The all-pairs shortest path problem [Dijkstra, 1959 consists of solving 



d{v,v') = min f(p) (1) 

for all vertices v, v' £ V, where V vv > is the space of all paths connecting v to v' in V, and f(p) is 
the path length, i.e., f{p) — Yli=i 1 e (Pi:Pi+i) where e(pi,pj) is the weight of the edge connecting 
Pi to pj, or oo if no such edge exists. 

A simple divide- and- conquer solution to (eq. [1} can be obtained by defining d(u,v,k) to be the 
shortest path between u and v containing at most k edges. This solution exploits the fact that 

l\ _ / e(u,v) if fc = 1 , , 

a[u, v,K)-^ min ^ ^ ^ k ^ + d ^ ^ k ^ otherwise W 

This allows us to solve the all-pairs shortest path problem via Algorithm [TJ which we requires 
Q(V 3 \og{V)) time (this is by no means the optimal solution, though it is this version to which our 
improvements apply). 



*The authors are with the Statistical Machine Learning Program at NICTA, and the Research School 
of Information Sciences and Engineering, Australian National University. Queries should be addressed to 
julian.mcauley@nicta. com. au. 
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Algorithm [TJ Line [9] requires that we solve a problem of the form 

$(a, 6) = min*i(a,x) + \E r 2(&, x) . (3) 

Although this appears to be a linear-time operation (in V), we note that it can be reduced to 
O(VV) (in the expected-case) if we know the permutations that sort v a and v^. The sorted values 
of Vb will be reused for every value of a, and likewise the sorted values of v a will be reused for every 
value of b. 

Lines EHU of Algorithm Q] are sometimes referred to as the "Funny Matrix Multiplication" 
problem: replacing (min, +) with (+, x) yields the traditional version of matrix multiplication. 
Kerr |Kerr, 1970] showed that it is fl(V 3 ) if only the operations min and + are allowed. We find 
that under reasonable conditions on v a and v;,, an expected-case sub-cubic solution exists, requiring 
only min and +. 

Algorithm 1 All-pairs shortest-path problem 
Input: a graph V 

1: for u E V do 

2: for v £ V do 

3: d(u, V, 0) := e(u, v) 

4: end for 

5: end for 

6: for % e {1 . . . flog^l} {k = 2 1 } do 
7: for iitVdo 
8: for v € V do 

9: d(u, v, i) — min x (d(u, x, i — 1) + d(x, v, i — 1)) {0(^)1 

10: end for 
11: end for {Q(V 3 )} 
12: end for {&(V 3 log{V))} 



2 Our Approach 

The following elementary lemma is the key observation required in order to solve (eq. [3]) efficiently: 

Lemma 1. If the p th smallest element o/v a has the same index as the q smallest element o/vf,, 
then we only need to search through the p smallest values of v a , and the q smallest values of v&; 
any values 'behind' these cannot possibly contain the smallest solution. 

This observation is used to construct Algorithm [U Here we iterate through the indices starting 
from the smallest values of v a and Vft, stopping once both indices are 'behind' the minimum value 
found so far (which we then know is the minimum). This algorithm is demonstrated pictorially in 
Figure [1] 

An upper-bound on the expected-case running time of Algorithm [5] is given by the following 
theorem: 

Theorem 2. The expected running time of Algorithm^ is 0(y/V). 
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Algorithm 2 Find i such that v a [i] + Vft [i] is minimised 

Input: two vectors v a and Vb, and permutation functions p a and pf, that sort them in increasing 

order (so that v a [p a [l]] is the smallest element in v ) 
l: Initialize: start := 1, end a := p a ; 1 [p( ) [l]], end b :— p' b 1 [p a [l]] {if end b — k, then the smallest 

element in v a has the same index as the k th smallest element in v^} 
2: best := p a [l], min := v a [6esi] + ~v b [best] 
3: if v [pi,[l]] +v b [p b [l]] < min then 
4: best := pb[i], min := v a [best] + Vf,[best] 
5: end if 

6: while start < end a do 
7: start := start + 1 

8: if v Q [p a [start]] + Vf,[p a [start]] < min then 

9: best :— p a [start] 
10: min :— ~v a [best] + ~v b [best] 
11: end if 

12: if p b 1 [p a [start]] < endt, then 
13: end b :— p^\p a [start]] 
14: end if 

15: {repeat Lines interchanging a and 6} 

16: end while {this takes expected time 0(y/V)} 
17: Return: best 



The cxpected-case running time arises under the assumption that v a and Vf, are uncorrelated. 
The running time approaches O(l) as v a and v& become increasingly correlated, and it approaches 
0(V) as v a and v& become increasingly anti-correlated. Algorithm [2] shall be analysed in detail in 
Section [3] 

Using Algorithm we can solve the all-pairs shortest path problem in 0(V 2 log(V)) in the 
expected-case, for graphs with edge-weights in R with no negative cycles. This is shown in Algo- 
rithm [3] For dense graphs, our method has worst-case performance &(V 3 log(V)), and best-case 
performance Q(V 2 \og 2 (V)). Our Algorithm requires Q(V 2 log(V)) memory. Also note that Al- 
goritm [5] can exploit sparsity in the graph structure: the algorithm may terminate as soon as it 
reaches entries with infinite weight - thus if only /(V) edges are viable, our algorithm has worst-case 
performance 0(V 2 f(V) log(V)) (meaning that it does not surpass Johnson's Algorithm on sparse 
graphs [Joh nson, 1977| ). 

2.1 Comparison to Existing Approaches 

To our knowledge, the only existing sub-cubic approach is due to |Alon et al., 1997] (for edge weights 
taking small integer values) ; our algorithm shall not surpass this per se, as it is not deterministic - it 
depends on the distribution of the edge weights, and it is certainly possible to adversarially generate 
graphs yielding worst-case performance. Our algorithm has best-case and worst-case performance 
of <d(V 2 \og 2 (V)) and Q(V 3 \og(V)) respectively; thus it does not surpass Floyd- Warshall on dense 
graphs in the worst-case. Unlike Floyd- Warshall it is able to exploit graph sparsity, though it does 
not have better worst-case performance than Johnson's Algorithm. In short, our algorithm does not 
improve upon existing solutions in the worst-case, though under reasonable conditions, it has lower 
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Figure 1: Left: The lists v a and v& before sorting. Right: Black squares show corresponding 
elements in the sorted lists (v a [p Q [z]] and V(,[pb[i]]); red squares indicate the elements currently being 
read (v a [p a [start]] and Vb[Pb [start]]). We can imagine expanding a gray box of size start x start 
until it contains an entry; note that the minimum is found during the first step. 
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Figure 2: (a) A permutation can be represented as an array, where there is exactly one non-zero 
entry in each row and column; (b) We want to find the smallest value of m such that the grey box 
includes a non-zero entry; (c) For the sake of establishing an upper-bound, we consider a shaded 
region of width f(V) and height m. 



complexity than existing algorithms. We shall see in Section [J] that our algorithm is significatly 
faster than Floyd- Warshall in practice, making it a viable solution to real-world all-pairs shortest 
path problems, despite its lack of worst-case guarantees. 

3 Asymptotic Performance of Algorithm [2] 

In this section we shall determine the expected-case running time of Algorithm [5] Algorithm [5] 
traverses v a and Vb until it reaches the smallest value of m for which there is some j < m for which 
m > Vb If M is a random variable representing this smallest value of m, then we wish to 

find.E(M). 

By representing a permutation of the digits 1 to V as shown in Figure [21 we observe that m is 
simply the width of the smallest square (expanding from the top left) that includes an element of 
the permutation (i.e., it includes i and p[i]). 

Simple analysis reveals that the probability of choosing a permutation that does not contain a 
value inside a square of size m is 

(V-m)\(V-m)\ 
P(M>m)= {V-2m)\V\ ■ (4) 
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Algorithm 3 All-pairs shortest-path problem in expected-case 0(V 25 \og(V)) 



Input: a graph V 
l: for u E V do 
2: for v E V do 

3: d(u, V, 0) := e(u, v) 
4: end for 
5: end for 



6: forie {l...riogV]} {k = 2 1 } 



do 



7: for iieVdo 

8: Pa{u) '■= permutation that 

9: Pb{u) ■— permutation that 

10: end for {Q(V 2 log(V))} 



sorts d(u, x, i — 1) 

sorts d(x,u,i- 1) {0{V\og{V))} 



11: for iieVdo 
12: for ueVdo 



13: y := Al$2[d(u,x,i - 1), 

14: d(u, v, i) := d(u, y, i — 1) 

15: end for 



d(ar,w,i-l),j» tt (u),p 6 (t;)) {O(W)} 
+ d(y,v,i- 1) 



16: end for {0(V 2 VV)} 

17: end for {0{V 2 VV\og{V))} 



This is precisely 1 — F(m), where F(m) is the cumulative density function of M. It is immediately 
clear that 1 < M < [V/2\ , which defines the best and worst-case performance of Algorithm [2] 

Using the identity E(X) — Yl'xLi P{X ^ x )i we can write down a formula for the expected 
value of M: 



Thus the expected-case running time of our all-pairs shortest path solver (assuming uncorrelated 
sub-problems) is Q(V 2 E(M ) log(F)). We show in the following section that E{M) E O(VV). 

3.1 An Upper Bound on E(M) 

Although (eq. [5]) precisely defines the running time of Algorithm [21 it is not easy to ascertain the 
speed improvement it achieves, as the values to which the summations converge for large V are not 
obvious. Here, we shall try to obtain an upper-bound on their performance, which we shall assess 
experimentally in Section [U In doing so we shall prove Theorem [2] 

Proof of Theorem^ Consider the shaded region in Figured] (c). This region has a width of /(V), 
and its height m is chosen such that it contains precisely one non-zero value. Let M be a random 
variable representing the height of the grey region needed in order to include a non-zero entry. We 
note that 




(5) 
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first m samples appear in the shaded region is 



P(M>m)=H(l-^l). (7) 



i=0 



Next we observe that if the entries in our V x V grid do not define a permutation, but we instead 
choose a random entry in each row, then the probability (now for M) becomes 



P{M>m)=(l-l^j (8) 

(for simplicity we allow m to take arbitrarily large values). We certainly have that P(M > m) > 
P(M > m), meaning that E(M) is an upper bound on E(M), and therefore on E(M). Thus we 
compute the expected value 

E{M)=Y j fl-l^iy. (9) 
This is just a geometric progression, which sums to V/f(V). Thus we need to find f(V) such that 

f(v) e o (-^-) . (10) 



\f(V). 

Clearly f(V) £ O(W) will do. Thus we conclude that 

E{M) £ 0(W). (11) 

□ 

We will show that this upper bound is empirically tight in the following section. 



4 Experiments 

4.1 Performance of Algorithm [2] 

For our first experiment, we compare the performance of Algorithm [5] to the naive linear time 
solution. We generate 2V uniform samples from [0, 1) to obtain the lists v a and v^. V corresponds 
to the size of the graph in question. The performance of Algorithm [5] is shown in FigureEJ the value 
reported is simply the value of start upon termination of the algorithm; this is compared to V itself, 
which is the number of elements read by the naive solution. The upper-bounds we obtained in the 
previous section are also reported, while the true expected performance (i.e., (eq. [SJ)). Visually, we 
find that our upper-bound is empirically very close to the true performance, suggesting that the 
bound is reasonably tight. 

4.2 Performance for Correlated Variables 

The expected-case running time of our algorithm was obtained under the assumption that the 
variables were uncorrelated, as was the case for the previous experiment. We suggested that we will 
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Figure 3: Performance of our algorithm and bounds. For K = 2, the exact expectation is shown, 
which appears to precisely match the average performance (over 100 trials). The dotted lines show 
the upper-bound, which appears to be extremely close to the average performance, indicating that 
the bound is reasonably tight. 

obtain worse performance in the case of negatively correlated variables, and better performance in 
the case of positively correlated variables; we will assess these claims in this experiment. 

We report the performance for two lists (i.e., for Algorithm^, whose values are sampled from 
a 2-dimensional Gaussian, with covariance matrix 



meaning that the two lists are correlated with correlation coefficient c. Performance is shown in 
Figured] for different values of c (c = 0, is not shown, as this is the case observed in the previous 
experiment). 

In real graphs, c shall be the correlation coefficient between p{u, x, i — 1) and p(x, v,i—l) (which 
is free over x). Unless c is equal to precisely —1 for all u, v, and i, we obtain a sub-cubic solution. 
Whether we observe positive, negative, or zero correlation will depend on the statistics of the graphs 
in question. 

4.3 Performance of Algorithm [3] 

Finally, we compare our algorithm to the divide- and-conquer solution of Algorithm [T] and to the 
popular Floyd- Warshall Algorithm Floyd, 1962] on dense graphs in R + . 

We generate dense graphs of size V with edge weights sampled uniformly in [0, 1). The perfor- 
mance of our algorithm, compared to Algorithm Q] and the Floyd- Warshall Algorithm is shown in 
Figure [51 We note that our algorithm is faster than Algorithm [1] after only V = 4, meaning that 
its computational overhead is negligible. It is faster than Floyd- Warshall after V ~ 90. 
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Figure 4: Performance of our algorithm for different correlation coefficients. The top three plots 
show positive correlation, the bottom three show negative correlation. Correlation coefficients of 
c = 1.0 and c = —1.0 capture precisely the best and worst-case performance (respectively) of our 
algorithm. 
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All-pairs shortest path 
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Figure 5: The running time of our algorithm compared to the divide-and-conquer solution of 
Algorithm [T] and the Floyd- Warshall Algorithm. The average of 10 trials is shown. All algorithms 
were implemented in Python. 

4.4 Conclusion 

We have presented an expected-case subcubic solution to the problem of Funny Matrix Multiplica- 
tion, resulting in an expected-case 0(V 2 5 log(V)) solution to the all-pairs shortest path problem. 
The running time of our method depends on the distribution of edge weights for the graph in 
question, though we achieve performance at least as good as the expectation under reasonable con- 
ditions. Our algorithm is significantly faster than Floyd- Warshall in practice, making it a viable 
solution to real-world all-pairs shortest path problems. 

Acknowledgements 

We would like to thank Pedro Felzenszwalb for alerting us to the link between inference in graphical 
models and the all-pairs shortest path problem. NICTA is funded by the Australian Government's 
Backing Australia's Ability initiative, and the Australian Research Council's ICT Centre of Excel- 
lence program. 

References 

[Alon et al., 1997] Alon, N., Galil, Z., and Margalit, O. (1997). On the exponent of the all pairs 
shortest path problem. Journal of Computer and System Sciences, 54(2):255-262. 

[Dijkstra, 1959] Dijkstra, E. W. (1959). A note on two problems in connexion with graphs. Nu- 
merische Mathematik, 1(1):269-271. 



9 



[Floyd, 1962] Floyd, R. W. (1962). Algorithm 97: Shortest path. Commun. ACM, 5(6):345. 

[Johnson, 1977] Johnson, D. B. (1977). Efficient algorithms for shortest paths in sparse networks. 
J. ACM, 24(1):1-13. 

[Kerr, 1970] Kerr, L. R. (1970). PhD Thesis. 



10 



